This invention relates generally to graphical image manipulation systems, and more particularly to a user interface for creating and editing electronic images of documents.
In interactive sketching and drawing systems, a single mouse/pen/stylus device serves both to input image material, through draw operations, and to perform command operations such as inputting a spatial selection gesture that selects out certain image objects to be operated on further, such as being moved, copied, or deleted. In existing systems, the user must indicate which type of input is to be performed by setting a “mode”. The necessity for setting the user interface mode becomes a distraction in the editing process and impedes the smooth flow of sketching and interactive editing tasks. Moreover, errors occur when users execute draw or selection/command gestures but neglect to ensure that the application is set to the mode corresponding to their intended action.
Two types of interactive drawing/sketching/editing applications are currently in use, both of which support creation of new image material, through draw operations, and selection and manipulation of existing material, through editing operations. The types of interactive applications are distinguished by the emphasis placed on “sketch” and “editing” operations. In an image “editing” program, selection and manipulation of image objects is the primary activity. Therefore, stylus or mouse interaction is designed primarily to interpret stylus input as selection gestures, and the default interpretation of mouse or stylus activity is selection of existing image objects for manipulation. Tools for drawing objects are provided by auxiliary command objects, usually menus.
In a “sketch” program, however, the primary activity is the “draw” operation. To facilitate the sketching process, it is important for users to be able to quickly execute a series of markings such as handwritten or sketched strokes, without having to perform a menu initiation command at every stroke. These programs are designed such that draw operations can be the default interpretation of mouse or stylus activity. The disadvantage to this type of program is that when priority is placed on draw operations, selection operations become demoted and require explicit menu choices or button clicks to invoke a selection, which impedes the smooth flow of multiple selection and manipulation operations.
One approach to resolving the question of the interpretation of mouse or stylus activity is the use of interface modes. Under one technique, the application is placed in “draw” mode or “select” mode by activation of a menu item, and is left in that mode until the user switches to another. Under another technique, the application is left in the “draw” mode unless the user depresses a mouse or pen button, which then places the application the “select” mode. In both of these cases, the user must be aware of the mode currently activated, and perform additional steps to switch modes as needed. Both editing and sketching programs may use modes in this manner.
U.S. Pat. No. 5,513,309 to Meier et al. titled “Graphic Editor User Interface for a Pointer-Based Computer System” discloses a graphical editor arranged to permit the user to edit selected graphic objects by highlighting the objects and moving them with editing handles. A bounding box is also drawn about the selected portions of the object. In various aspects of the invention, the user is permitted to edit the object by executing specific actions, including resizing, duplicating, distorting and moving either the entire object or only selected portions. After any of the editing operations is performed, the display is updated to reflect changes made during the editing step.
U.S. Pat. No. 5,523,775 (1996) to Capps titled “Method for Selecting Objects on a Computer Display” teaches a method for selecting objects on a screen of a pen-based computer system. The user's intention to enter the selection mode is indicated by the user's performing a special action with the stylus, such as holding the stylus in one place for a minimum amount of time, tapping the stylus immediately prior to and in the same location the highlighted gesture is to begin, or drawing a small circle. Each of these approaches is prone to user error. For example, a user may place the stylus down while thinking about the shape of the stroke to be drawn and unexpectedly be placed in a “highlight” (selection) mode. Similar problems occur with the “tap-and-a-half” and “small circle” methods described in Capps. Like other existing methods in this field, under the Capps invention the user is required to declare in advance of their stroke that the stroke is to be interpreted in terms of “highlight” (select) mode.
U.S. Pat. No. 5,953,735 to Forcier titled “Script Character Processing Method and System with Bit-Mapped Document Editing” teaches a pen-based processor which enables a user to input and edit script like a text-based computer, but retains a resemblance to the use of a pad and pencil. The system enables the input, editing, and other manipulation of glyphs, including handwritten script, ASCII text, bit-mapped images and drawings in a common document, using a compatible internal representation of the data and a set of user control functions. These functions are invoked using a two-step gesture method to avoid confusion between strokes and command gestures and also to allow use of similar gestures for different functions within the same and different contexts. Again, the intended mode must be specified by the user before a stroke is initiated. The system infers from customary user writing conventions that certain relationships of data are to be preserved, including delineation of words and paragraphs, and maintains the relationships, subject to user override, during editing. The display document is formatted to contain lined or unlined areas of glyphs that can be edited such as by mapping each glyph in the image lines to a machine editable line space, including insertion of a moving space into pre-existing document text and word wrapping.
The present invention offers a new user interface technique employed in an interactive digital ink sketch/drawing/editing program. Instead of requiring the user to set input mode, such as “Draw Mode” or “Select Mode” initially and then interpreting gestures according to the specified mode, the present invention postpones the decision as to which mode the user intends until the gesture is complete. If the type of gesture can be inferred from its shape and timing characteristics in relation to data structures describing individual image objects and groupings of these objects, then the gesture is invoked as that type. Only if the gesture is ambiguous is the user presented with the opportunity to choose between the modal options.